Handbook on Measurement, Assessment, and Evaluation in Higher Education by Denison D. Brian. Secolsky Charles

Author:Denison, D. Brian.,Secolsky, Charles.
Language: eng
Format: epub
Publisher: Taylor & Francis Ltd

The creation of PLDs can be a very time-consuming enterprise and add up to a full day to the process of setting cut scores. For that reason, it may be preferable to convene a panel of experts expressly for the purpose of creating the PLDs prior to the cut score session, so that this critical part of the process is given full attention. Then, during the process of setting cut scores, panelists are given the prepared PLDs and provided an opportunity to discuss, edit, and refine them. The panelists convened to develop the PLDs may overlap in part or in full with those who serve on the standard setting panel.

The performance level descriptors will be a major part of the documentation provided to the policymaker and in the final report. The final set of performance level descriptors provides the meaning in words of the numeric cut score that will be set and adds to the validity of the standard setting process and the resultant cut score (Perie, 2008; Hambleton, 2001). For additional information on writing performance level descriptors and examples that have been used in standard setting studies, see Perie (2008), Cizek and Bunch (2007), Hambleton (2001), or Hansche (1998). For sample PLDs that could be used with the SAT Reasoning Test mathematics section, see Morgan (2006).

Define the Borderline Examinee

The PLDs describe the knowledge and skills for all examinees in a given performance level. For example, in a placement setting context, PLDs may describe the characteristics of (a) examinees who will succeed in a full-credit, entry level course and (b) examinees who will need remediation.

Within the first level, there will be examinees who have so much knowledge and skills that they will likely perform at the top of their class. There will also be examinees who perform at an average level within that category. Of interest in the standard setting process are those examinees who are on the borderline between the lower and higher categories. They are just good enough to be placed into an entry level course.

A key step in the standard setting process is to work with the panelists to define the borderline examinee. The goal is to discuss the PLDs and specify the level of knowledge and skills called for in that level that the borderline examinee will possess. The borderline examinee will not know or be able to do all of the things listed in the PLD for a given level, so discussion needs to take place so that the nature of the knowledge and skills can be refined. In essence, this definition is a second set of PLDs that are specific to those students at the border between the performance levels. It is helpful to have the facilitator or an observer record the definition of the borderline examinee so that this material can be copied and distributed to panelists for their reference as they are making standard setting judgments.

The process of setting cut scores should not proceed until the panelists feel comfortable with the definition

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.